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Abstract. The communicative language teaching approach has dominated 
English teaching and learning since the 1970s. In Taiwan, standardized and high- 
stakes English tests also put focus on the assessment of learners’ communicative 
competence. While the test contents change, the modifications teachers made are 
superficial rather than substantial. A comparative corpus-based analysis of the 
English test items in Senior High School Entrance Exams (SHSEE) and curriculum- 
based English textbooks was conducted to provide more valid information for 
syllabus design, language instruction, and materials development. Two major 
corpora were compiled: an exam corpus, consisting of English test items in SHSEE 
from 2001 to 2014 and a junior high school English textbook corpus. AntConc and 
Readability Test Tool were employed to analyze the frequency of occurrence of 
the marked structures (relative, adverbial, and passive clauses) in the two corpora. 
The results showed rare occurrence of the marked structures in SHSEE but much 
higher frequency in textbooks. Teachers might be textbook-oriented, rather than 
test-oriented. It was suggested, based on the corpus-based analysis of the native 
speakers’ use, that a grammar list with finer guidelines for the national curriculum 
and textbook writers should be built. It could provide teachers easily accessible 
reference in selecting grammatical patterns for teaching. 
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1. Introduction 

Since the 1970s, the communicative language teaching approach has dominated 
the English language teaching and learning in all contexts. Test writers for SHSEE 
have intended to bring about the positive changes by testing learners’ English 
communicative competence in Taiwan. However, teachers still devote a substantial 
amount of class time to traditional activities such as textbook explanation or 
exercises on grammar (Pan, 2011). They also have a propensity to prepare learners 
for English tests through abundant test-preparation instruction. Although most 
studies indicate that teachers tend to teach for the test, several studies state that 
tests do not significantly infiuence teaching (e.g. Green, 2007); the modifications 
teachers have made seem to be superficial rather than substantial. Therefore, in this 
study, we carried out a comparative corpus-based analysis of the English test in 
SHSEE and curriculum-based English textbooks. This research aimed to examine 
whether there was a mismatch between what is being taught and tested. The study 
was designed to address two research questions. First, what is the frequency of 
occurrence of marked structures in the English exam in SHSEE and the curriculum- 
based English textbooks? Second, what is the backwash effect of the English exam 
in SHSEE on teachers’ instruction? 

2. Method 

2.1. Corpus selection 

Two main corpora were compiled for the current study. One was CAP corpus, 
which consisted of all the English test items in SHSEE from 2001 to 2014. The 
other one is textbook corpora, including Kang-Xiuan Corpus and Han-Lin Corpus, 
two major versions of junior high school English textbooks in Taiwan. The marked 
structures (relative, adverbial, and passive clauses) were chosen for the study 
because they were difficult for Taiwanese learning English as a foreign language 
(EEL) (Chang, 2008). Moreover, teachers firmly believed learners had to master 
these structures to get higher scores in the entrance exam. 

2.2. Data analysis 

The corpora were analyzed by using AntConc 3.4.3 (Anthony, 2014) and 
the Readability Test Tool (Simpson, 2014). The frequency of occurrence of 
the marked structures in the CAP, Kang-Xiuan, and Han-Lin Corpora was 
counted. Then, a comparative corpus-based analysis of the collected data was 
conducted. 
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3. Discussion 

Table 1 showed that the total number of the relative clauses occurring in the CAP 
corpus is 249. About 98% of them were designed for recognition (reading) and 
only 2% were for production (writing). The adverbial clauses occurred 106 times 
but only 1.88% of them were tested for production. The passive clauses appeared 
87 times but only 13 occurrences were tested for production. Most of the marked 
structures were tested for recognition (97%), not for production (3%). The rare 
occurrence of these complex structures in the English exam was in line with Ellis’s 
(2012) proposition that frequent and unmarked structures should be acquired first 
in L2 acquisition. 


Table 1. Frequency of the relative, adverbial, and passive clauses in the CAP 
corpus 




CAP Corpus 


Grammatical Patterns 

Tokens for Reeognition 

Tokens for Produetion 

Total Tokens 

A. Relative 
1 . that 

126 

3 

129 

2. who 

65 

2 

67 

3. whieh 

10 

1 

11 

4. whose 

1 

0 

1 

5. where 

18 

0 

18 

6. how 

6 

0 

6 

7. why 

17 

0 

17 

B. Adverb 
1. if 

87 

1 

88 

2. although 

8 

1 

9 

3. though 

8 

8 

9 

C. Passive 
1. by 

79 

8 

87 

Total 

410 

16 

426 


Teachers in Taiwan invested lots of class time to teach these infrequent marked 
structures, which was believed to help students get higher test scores (Chang, 
2008). However, the results revealed the English exam in SHSEE contained a 
very low proportion of the marked structures. Test takers were rarely required to 
produce these marked structures, except for recognition. Teaching for the test, at 
most, means teachers adhered to the test format, rather than the test construct. Such 
conflict might be attributed to teachers’ incomplete understanding of the nature of 
the exam, inadequate training, or lack of professional background. 

Examining the English textbooks provided another explanation for teachers’ 
overemphasis on marked structure instruction. Table 2 showed the result of 
the frequency occurrence of the marked structures in the two most popular 
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versions of junior high school English textbooks. The frequency of the complex 
sentence patterns in Kang-Xuan and Han-Lin corpora were 13.63% and 31.77%, 
respectively. There was an overemphasis on rarely used marked structures in the 
English textbooks. The results implied teachers might be textbook-oriented, rather 
than test-oriented. The findings showed the backwash effect, i.e the effect of the 
examination on classroom methodology and selection of teaching materials, to be 
limited, although the test construct of the English exam in SHSEE showed marked 
changes over the past 14 years in Taiwan. 


Table 2. Frequency of the relative, adverbial, and passive clauses in the textbook 
corpora 



CAP 

Kang-Xuan 

Han-Lin 

Grammatical Patterns 

Tokens Non-tested tokens 

Tested tokens Tokens 

Tokens 


A. Relative 
1. that 

129 

126 

3 

89 

212 

2. who 

67 

65 

2 

24 

65 

3. which 

11 

10 

1 

11 

31 

4. whose 

1 

1 

0 

1 

3 

5. where 

18 

18 

0 

7 

13 

6. how 

6 

6 

0 

6 

18 

7. why 

17 

17 

0 

5 

7 

B. Adverb 
1. if 

88 

88 

0 

61 

99 

2. although 

9 

8 

1 

10 

19 

3. though 

9 

8 

1 

1 

4 

C. Passive 
1. by 

87 

79 

8 

51 

118 


Total 

426 

410 

16 

266 

589 

Total word tokens 

55440 



49050 

48072 

Total sentence numbers 

6551 



7104 

7362 


4. Conclusion 

The results indicated a rare occurrence of the marked structures in the English 
exam in SHSEE; there was, however, an overemphasis placed on them in the 
surveyed textbooks. Such information can be vital in designing curriculum, and 
teachers should use this information to effectively direct their pedagogical focus, 
since it is not sufficient to change exams. As Cheng (2005) mentioned, an exam 
on its own cannot bring about change if the educational system has not adequately 
prepared the teacher. Teachers and language testers should try to make a match 
between what is tested and what is taught. Chang (2008) contends that positive 
test effects usually occur when there is a match between test content and curricula. 
To facilitate EEL learners’ acquisition, teachers should avoid overloading learners 
with infrequent and marked structures. Moreover, teachers should not “blindly” 
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follow the textbooks. Two pedagogical implications were suggested. First, there is 
a need to develop a grammar list with clear guidelines for the national curriculum 
and textbook writers. Second, a distinction should be made between production 
and recognition structures in the junior high school English textbook. This would 
provide an easily accessible reference to guide teachers in selecting grammatical 
patterns for teaching. 
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